D  M  S  T  T  I  A  C 

Defense  Modeling,  Simulation  and  Tactical  Technology 
Information  Analysis  Center 


DMSTTIAC  TA-97-02 


An  Assessment  on  Modeling  &  Simulation  of 
Infrared  Sensor  Systems 

A  Modeling  and  Simulation  Assessment 


Mitchell  W.  Douglas 

Associate  Engineer 
NT  Research  Institute 


Published  by: 

DMSTTIAC 

I  IT  Research  Institute 

7501  South  Memorial  Parkway,  Suite  1 04 

Huntsville,  Alabama  35802 


Approved  for  Public  Release; 
Distribution  is  Unlimited 


June  1997 


DTIC  QUALITY  INSPECTED  4 


19990604  030 


INTRODUCTION 


Modem  military  systems  being  fielded  are  becoming  increasingly  more  complex  and 
expensive,  and  challenge  traditional  training,  analysis,  and  test  &  evaluation 
methodologies.  As  a  result,  there  is  more  and  more  pressure  to  use  modeling  and 
simulation  throughout  the  development  cycle  of  these  military  programs.  The  need  for 
modeling  and  simulation,  especially  Distributed  Interactive  Simulation  (DIS),  is 
becoming  more  apparent  for  operational  test  and  evaluation  as  well  as  for  defining 
military  training  requirements.  In  the  process  of  system  design  and  development,  we 
depend  more  on  the  high-fidelity  deterministic  and  statistical  modeling  and  simulation 
approaches. 

One  thing  that  needs  to  be  kept  in  mind  is  the  fact  these  simulations  are  just  what 
they  are,  mathematical  models  of  systems  interacting  to  play  out  a  scenario  and  provide 
results.  These  results  are  analyzed  and  decisions  are  made.  Therefore  in  order  to 
interpret  the  results,  it  is  imperative  that  the  mathematical  models  represent  the  system 
and  that  the  input  data  is  understood.  As  we  become  more  dependent  on  modeling  and 
simulation,  the  approaches  taken  in  generating  the  “synthetic  real  world”  must  take  into 
consideration  the  anticipated  advancements  in  the  system  being  modeled.  The  objective 
of  this  assessment  is  to  look  at  some  of  the  methodologies  used  in  modeling  and 
simulation  of  infrared  (IR)  search-and-target  acquisition  systems,  to  address  the 
generation  of  synthetic  IR  imagery,  and  to  identify  issues  associated  with  modeling  and 
simulating  IR  imagery  in  real  time  as  3rd  generation  IR  technology  matures. 

PROGRESSION  OF  IR  SENSORS  AND  PERFORMANCE  MODELING 

Over  the  past  30  years,  IR  sensor  technology  has  advanced  tremendously  in  the  areas 
of  night  vision,  target  acquisition,  and  missile  seeker  applications,  from  single  element 
detectors  with  a  dedicated  preamp  performing  a  sequence  of  horizontal  scans  to  multiple 
detectors  performing  a  parallel  scan.  Soon  after,  integrated  circuit  technology  along  with 
advances  in  thin  films  have  allowed  for  the  development  of  linear  detector  arrays  with 
charge  coupled  device  (CCD)  multiplexer  and  preamp  performing  parallel  scan.  In  more 
recent  years,  2nd  generation  sensor  technology  has  advanced  the  development  into  high- 
density  two-dimensional  focal  plane  arrays  (FPA).  Figure  1  shows  this  evolution  of  IR 
sensor  technology. 
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With  the  complexities  of  improved  IR  sensors/seekers  came  the  complexities  in 
modeling  to  describe  the  performance  of  those  systems.  In  general,  the  modeling  has 
progressed  from  the  non-imaging  to  the  imaging  types  of  IR  sensor/seeker.  Of  significant 
importance  to  the  modeler  is  the  characterization  of  the  background  clutter.  Therefore,  as 
technology  and  modeling/simulation  capability  improve,  advances  in  the  characterization 
of  the  background  clutter  have  been  required. 

Early  IR  System  Modeling 

The  earliest  weapon  systems  implementing  IR  target  acquisition  and  guidance  were 
designed  for  antiaircraft  missiles.  These  systems  were  typically  a  single  element  detector 
utilizing  a  reticle  with  a  nutating  scan  to  generate  guidance  signals.  They  were  lock-on- 
before-launch  systems,  which  depended  on  a  high  enough  signal-to-noise  ratio  for  target 
lock.  The  reticle  extent  and  the  scan  pattern  resulted  in  rather  large  fields  of  view  (2-3 
degrees)  and  even  though  reticle  design  attempted  to  suppress  clutter  effects,  the 
background  clutter  (both  terrain  and  cloud)  could  produce  a  signal  which  was 
significantly  higher  than  the  signal  from  a  point  source  target.  The  contribution  of  clutter 
to  the  overall  signal  was,  in  large  part,  ignored  and  the  target  acquisition  modeling 
process  normally  assumed  a  “blue  sky”  background  situation.  The  important  variables  in 
modeling  (equation  1)  these  systems  were  the  system  sensitivity  (Noise  Equivalent 
Irradiance  (NEI)),  “effective”  target  radiant  intensity  (Jeff),  range  (R)  and  atmospheric 
transmission  (xa).  If  clutter  was  considered  at  all,  it  was  introduced  as  a  multiplier  to  the 
seeker  signal-to-noise  (S/N)  required  for  tracking. 


- — .  NEI 
R2  N 


(1) 


The  next  generation  of  IR  sensor/seeker  systems  was  considered  “pseudo”  imaging. 
By  scanning  the  system  field  of  view  with  one  or  more  detectors,  an  image  of  the  scene 
can  be  created.  By  storing  the  signal  response  of  each  detector  during  each  dwell  time  of 
the  scan  cycle,  the  scene  can  be  processed  as  an  image.  By  interrogating  the  stored 
image,  guidance  control  commands  can  be  initiated  autonomously  or  by  man  in  the  loop. 
Now,  the  complexity  of  modeling  the  sensor/seeker  performance  is  growing. 

Background  clutter  now  becomes  a  variable  in  the  model  equation  and  must  be 
represented  as  accurately  as  possible. 


Jeff  ‘ Tg- = — .  4neF2  +  NEC2 

R2  N 


(2) 


Equation  2  represents  the  case  where  the  target  is  no  longer  against  a  “blue  sky.” 
NEC  represents  the  background  equivalent  irradiance.  This  process  assumes  normal 
distribution  of  the  background  clutter;  therefore,  it  can  be  root  sum  squared  with  the 
sensor  internal  noise.  There  is  much  data  available  which  characterizes  the  background 
in  terms  of  the  RMS  level  in  °C.  Although  it  is  recognized  that  this  is  a  simplistic 
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characterization  of  the  very  complex  background,  its  use  is  easily  adapted  to  modeling 
with  results  comparable  to  test  data.1  Modeling  has  evolved  from  the  statistical  efforts  of 
test  and  data  collection  to  a  more  deterministic  higher  fidelity  modeling  approach,  which 
includes  specific  sensor  system  parameters,  spatial  filters,  and  acquisition  and  tracking 
algorithms.  The  clutter  input  for  these  models  can  utilize  actual  or  simulated  imagery 
developed  from  statistical  parameters. 

The  next  generation  of  sensor  developments  was  full  imaging  utilizing  staring  focal 
plane  arrays  (FPA).  These  sensor  systems  have  created  the  most  controversy  regarding 
the  different  methods  to  model  performance  against  targets  in  a  variety  of  backgrounds 
(i.e.,  deterministically  or  statistically).  The  modeling  approach,  as  shown  in  Figure  2,  is 
based  on  that  initially  configured  for  man-in-the-loop  (MIL)  target  detection  and 
acquisition  systems.  For  MIL  systems  the  performance  is  based  on  minimum  resolvable 
temperatures  (MRT)  and  minimum  detectable  temperatures  (MDT).  These  can  be 
derived  from  models  which  characterize  the  target  as  a  delta  temperature  through  the 
optics,  FPA,  electronics,  signal  processor,  display,  and  observer.  Ultimately  it  is  the 
observer’s  capability  of  distinguishing  the  target  from  the  background  based  on  the 

characteristics  of 
the  IR  sensor  that 
characterizes 
performance.  The 
display  and 
observer  aspects  are 
unique  to  MIL 
modeling  and  are 
not  generally 
considered  for 
autonomous 
performance 
modeling. 


Figure  2.  Modeling  of  Imaging  IR  Sensor  Systems 


Autonomous  modeling  can  be  explicitly  scene  based,  with  either  real  or  synthetically 
generated  imagery,  or  statistically  based  using  parameters  which  represent  clutter  scenes. 
The  most  accurate  method  of  modeling  clutter  is  using  actual  scene  imagery. 

Synthetically  generated  scenes  can  also  provide  accurate  clutter  data  for  modeling  but 
synthetically  generated  scenes  are  only  as  good  as  their  input  data  or  the  fidelity  of  the 
scene  generator.  The  disadvantage  of  this  method  is  that  it  is  computationally  intensive 
and  requires  elaborate  scene  data  collection,  which  cannot  realistically  cover  all  regions. 
An  alternate  type  of  modeling  incorporates  statistical  models  of  clutter.  This  allows  for 
faster  execution  times  and  a  wider  variety  of  clutter  categories.  The  disadvantage  with 
statistical  clutter  modeling  is  that  it  is  often  too  general  and  does  not  consider  the  detail  of 
background  clutter  that  would  be  utilized  in  a  scene-based  imaging  simulation. 
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Progression  of  Clutter  Modeling 

Statistical  background  clutter  modeling  has  evolved  from  the  simple  subjective  to 
the  more  complex.  Initially,  clutter  was  characterized  as  “high”  or  “low”  and  modeled 
with  an  appropriate  gain  applied  to  the  signal-to-noise  ratio.  As  the  technology  moved 
toward  imaging  systems  clutter  became  an  important  element  in  modeling  performance. 
At  that  time  (70’s  -  80’s)  an  adequate  methodology  was  not  developed  to  quantify  its 
effect.  During  the  80’s  the  methodology  adopted  to  model  clutter  made  use  of  a 
background  temperature  with  an  associated  standard  deviation.  The  standard  deviation  of 
the  background  would  be  converted  to  an  equivalent  rms  power  density  and  statistically 
summed  with  the  internal  noise  of  the  sensor  system  as  a  noise  equivalent  clutter 
(equation  3). 


NEC-<o-fcL-a0-dT^G  (3) 

Jjff  ‘±a  =  S_  .  ~  J^-2 

R 2  N 

This  methodology  was  an  improvement  over  the  subjective  type  of  clutter 
characterization;  however,  there  were  still  several  notable  inadequacies.  F or  instance,  the 
critical  spatial  elements  of  clutter  were  not  being  modeled  directly,  and  there  was  no 
traceability  to  the  actual  spatial  content  of  the  scenes.  There  were  also  discrepancies 
between  modeled  performance  and  field-tested  performance.  Presently,  statistical  clutter 
modeling  techniques  have  grown  out  of  efforts  to  include  both  the  power  and  spatial 
effects  of  IR  clutter. 


Figure  3.  Evolution  of  Characterizing  Clutter 
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Through  analysis  of  these  several  forms  of  PSD  representations  of  clutter.  Model  A 
appears  to  most  closely  fit  the  PSD  curve  of  actual  clutter  imagery,  particularly  in  the 
higher  spatial  frequencies  important  for  small  target  detection.3  Model  A  includes  spatial 
characteristics  of  clutter,  unlike  previous  statistical  models  that  consider  only  the  mean 
and  standard  deviation  of  background  temperature.  Model  A  is  computed  in  the 
frequency  domain  and  approximates  the  PSD  of  the  scene.  The  parameters  that 
characterize  this  type  of  model  include  standard  deviation  of  background  temperature 
(ct0),  correlation  length  (Ro),  and  the  power  rolloff  exponent  (m).  An  example  of  these 
parameters  defined  for  several  environment  types  is  shown  in  Table  1.  In  general,  o0 
represents  the  deviations  of  scene  temperature,  or  the  background  uniformity.  Ro 
describes  the  spatial  characteristics,  or  size  of  the  background  clutter  components,  and  m 
indicates  the  quantity  of  high  spatial  frequency  clutter  relative  to  the  low  spatial 
frequency  clutter. 

f  =  spatial  frequency  (fx,  fy)  C0  =  km  sin(k7t/m)  Ro  c0  PSD(f)  = - - — 

f0  =  l/27tRo  k  =  1 .75  for  m  <  2.5  i  + 

k  =  2  .0  for  2.5  <  m  <  3.5  f0 


Table  1.  Initial  Model  A  Library 


BACKGROUND 

TYPE 

ROLL  OFF 

m 

CORRELATION 

LENGTH 

Ro  (M) 

STD.  DEVIATION 

cro  (°K) 

Open  Ocean 

2.9 

5.3 

0.86 

Overcast  Clouds 

2.6 

8.0 

2.8 

Snow  Covered 

3.2 

16 

1.5 

Forest 

Plowed  Field 

2.5 

23 

3.1 

Forest 

2.8 

5.3 

2.8 

Semi  Arid 

2.4 

8.0 

7.3 

Shrubs/Hills 

Urban 

2.2 

3.2 

4.9 

LEVELS  IN  MODELING  IR  SYSTEMS 

The  modeling  of  IR  sensor  systems  in  realistic  scenarios  has  become  extremely 
complex.  This  is  due  to  the  high  resolution  nature  of  modem  sensors,  the  complex  nature 
of  filtering  and  signal  processing  algorithms,  and  the  interrelated  nature  of  the  large 
number  of  parameters  that  affect  the  target  signature,  the  background’s  clutter  signature, 
the  engagement  scenario  and  the  intervening  environments.  Deterministic  models  are 
useful  and  necessary  for  detailed  analysis  of  IR  sensor  system  development  and 
refinement.  However,  limitations  exist  for  broad  and  early  comparative  analyses. 
Empirical  or  statistical  models  are  attractive  and  needed  for  trade-off  assessments  as  well 
as  test  and  evaluation.  But,  these  models  require  additional  sophistication  in 
accommodating  the  effects  of  background  clutter  and  need  an  extensive 
verification/validation  process  with  actual  IR  sensor  test  data. 
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Simplistic  models  may  or  may  not  be  sufficient  for  predicting  the  performance  of  full 
imaging  systems.  Some  would  agree  that  full  imaging  IR  systems  should  be  evaluated 

using  a  high  fidelity  model  that 
includes  the  exact  representation  of 
the  optics,  FPA,  non-uniformity 
correction  and  signal  processing 
algorithms.  Others  may  claim  that 
a  less  detailed  generic  model  is 
sufficient  for  system  evaluation. 
Both  are  correct.  The  dilemma  of 
choosing  a  modeling  approach  is  in 
understanding  the  merits  of  the 
model.  Figure  4  illustrates  this 
dilemma.2 


The  high  fidelity,  thorough,  point-to-point  model  process  is  well  understood  and 
useful.  In  general,  it  requires  a  very  extensive  input  data  requirement,  a  long  run  time 
(even  with  current  computing  technology),  and  can  produce  differing  results  with  small 
input  data  changes.  This  process  is  extremely  valuable  in  developing  the  exact  system 
figures  of  merit  and  signal  processing  algorithms.  High  fidelity  models  are  component- 
level,  physics-based,  and  are  typically  used  to  generate  the  performance  statistics  for 
input  to  lower  fidelity  models.  An  example  of  a  generic  high  fidelity  model  of  a 
submunition  implementing  an  IR  sensor/seeker  is  shown  below  in  Figure  5. 


Figure  5.  Generic  High  Fidelity  Weapon  System  Model 


Figure  4.  The  Modeling  Path 


High  Fidelity  Modeling  and  Simulation 
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This  is  a  general  representation  of  the  submunition  level  model.  It  is  one  system 
consisting  of  integrated  components  and  subsystems.  The  input  into  this  system  model  is 
an  IR  scene  with  one  or  multiple  targets,  weather  and  environmental  conditions.  The 
results  of  modeling  end-to-end  (simulation)  are  typically  a  list  of  statistics,  such  as 
probability  of  detection  (Pd),  probability  of  acquisition  (Pacq),  probability  of  track  (Ptrk), 
probability  of  hit  (Phit),  and  probability  of  kill  (Pk).  However,  each  subsystem  and 
integrated  component  has  an  output  serving  as  input  into  the  next  model.  For  example, 
the  IR  sensor  model  consists  of  optics,  detector,  and  electronics  models.  The  sensor 
model  takes  scene  radiance  data,  with  spatial  variation,  as  input  and  outputs  analog 
voltage  levels.  The  analog  signals  are  converted  into  digital  intensity  levels  and  stored  in 
an  indexed  array  for  input  into  the  guidance  and  control  model,  which  may  contain  the 
automatic  target  recognition  algorithms.  The  important  thing  to  keep  in  mind  is  that 
confidence  in  the  output  data  is  based  on  the  fidelity  of  the  models  and  the  representation 
of  the  synthetic  input  scene. 

The  U.S.  Army  Night  Vision  and  Electronic  Sensors  Directorate  (NVESD)  has 
focused  much  attention  on  the  appropriate  characterization  of  IR  sensors  in  simulation, 
and  the  synthetic  IR  scenes  that  drive  them.  Different  IR  sensors  have  distinctly  different 
levels  of  search  and  target  acquisition  performance  (different  sensitivity,  resolution,  etc.) 
NVESD’s  research  has  shown  that  the  realistic  simulation  of  IR  search  and  target 
acquisition  (STA)  systems  requires  two  things:  appropriately-characterized  sensor 
effects,  and  physically  reasonable  target  and  clutter  background  scenes  synthesized  to  at 
least  the  resolution  limit  of  the  sensor.  For  some  modeling  techniques,  the  sensor  is 
characterized  by  a  unique  modulation  transfer  function  (MTF)  and  noise  phenomenon 
that  defines  just  how  much  spatial  and  amplitude  detail  it  can  present  to  the  viewer. 

There  is  a  concern  that  less  simulated  scene  fidelity  than  the  sensor  can  actually  resolve 
frequently  results  in  a  mischaracterization  of  simulated  sensor  performance.  This  will 
result  in  a  false  assessment  of  performance. 

NVESD  presented  a  paper  at  the  1997  Spring  Simulation  Interoperability  Workshop 
addressing  the  fidelity  requirements  for  IR  simulation.4  In  this  paper,  fidelity  is  defined 
as  the  amount  of  spatial  and  amplitude  signature  resolution  in  the  synthesis  of  the  IR 
scene.  Spatial  resolution  can  be  increased  in  real-time  computer  graphics  by  increasing 
the  number  of  object  model  facets  or  by  increasing  the  resolution  of  the  applied  texture. 
Amplitude  resolution  refers  to  the  number  of  bits  representing  pixel  intensity  in  the 
output  frame  buffer  where  the  scene  geometry  is  rendered,  and  is  typically  8, 10,  or  12 
bits.  Since  increasing  fidelity,  usually  spatial  resolution,  generally  reduces  the 
sustainable  frame  rate,  it  is  very  useful  to  define  an  objective  measure  of  the  fidelity 
specifying  a  minimum  resolution  required  to  support  IR  sensor  simulation  to  some 
acceptable  level. 

NVESD  has  proposed  just  such  a  minimum  resolution  model,  referred  to  as  the 
MRT/MRC  and  MDT  -  Based  Fidelity  model.5  Since  all  computer  image  generators 
have  finite  resources  (texture  memory,  CPU  processing  power,  raster  managers,  graphics, 
pipelines,  etc.),  this  model  can  be  used  to  put  an  upper  bound  on  how  much  fidelity  to 
apply  as  a  function  of  range.  The  basis  of  this  model  is  that  the  atmosphere  and  sensor 
combine  to  form  a  filter  that  places  an  upper  limit  on  the  amount  of  spatial  resolution  a 
simulated  imaging  IR  sensor  system  can  resolve.  MRT  and  MTF  limited  curves  are 
generated  by  attenuating  the  background  contrast  detail  through  the  atmosphere  as  a 
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function  of  range  to  produce  an  apparent  contrast.  These  apparent  contrasts  are  then  used 
to  find  their  corresponding  spatial  frequency  “cutoffs”  from  the  sensor  MRT  and  MTF 
curves  (fc).  These  fc’s  are  then  mapped  to  an  associated  resolution  by  assuming  worst- 
case  normal-presented  background  to  the  sensor  field  of  view  (FOV),  and  applying  the 
Nyquist  sampling  criteria: 


^max/min  =  Range  12  fc  KV 

Resolution  in  excess  of  Rmax  will  be  completely  attenuated,  while  resolution  in  excess  of 
Rmin  will  not  be  resolvable. 

In  cases  where  the  resolution  requirements  are  high,  all  of  the  resolution  need  not 
necessarily  be  implemented  by  increasing  facets  (polygon  count).  Polygon  vertex 
emission  may  be  used  to  represent  the  mean  radiant  emittance,  while  high-resolution 
texture  representative  of  the  surface  thermal  variation  might  be  used  in  addition  to  vary 
this  mean  emittance.  The  standard  deviation  of  this  variation  would  be  representative  of 
the  surface  optical  and  thermophysical  property  distribution.  Application  of  texture  is  a 
practical  way  to  meet  the  MRT-Based  resolution  requirements  and  enhance  realism  in 
most  cases.  However,  texture  cannot  always  make  up  for  lack  of  geometric  detail.  Such 
terrain  detail  is  obviously  important  in  representing  lines-of-sight  (LOS)  correctly,  and  in 
representing  hill-lines  and  skylines  appropriately.  Meter  and  sub-meter  detail  is  often 
required  to  represent  terrain  realistically;  however,  it  results  in  terrain  databases  of 
millions  of  facets.  Clearly,  to  render  all  of  these  terrain  facets  and  update  them  with 
appropriate  emissive  components  for  every  frame  time  requires  some  innovative 
techniques.  This  is  the  main  issue  with  implementing  synthetic  IR  scenes  in  a  DIS 
environment  where  the  simulation  is  playing  out  in  real  time. 

Distributed  Modeling  and  Interactive  Simulation 

Distributed  modeling  and  interactive  simulation  are  becoming  more  and  more 
complex  and  are  requiring  detailed  modeling  of  the  various  systems  and  subsystems 
aboard  an  entity.  Currently,  the  state  of  an  entity  in  DIS  is  represented  by  the  Entity  State 
Protocol  Data  Unit  (ESPDU)  and  does  not  provide  enough  physical  state  parameters  for 
proper  representation  in  a  dynamic  situation,  specifically  IR  signature  and  background 
scenes.  Typically,  an  IR  signature  of  an  entity,  such  as  an  aircraft,  may  be  represented 
under  a  steady  state  condition  by  which  the  IR  signature  is  derived  from  a  database  using 
two  parameters,  airspeed  and  altitude.  Consider  the  aircraft  in  a  dynamic  state,  such  as  a 
dogfight.  The  aircraft  is  pulling  higher  G’s  in  a  tight  turn  at  a  lower  altitude  and  slower 
airspeed.  One  would  expect  to  see  the  engines  at  full  throttle,  thus  presenting  a  different 
IR  signature  than  would  be  expected  under  a  steady  state  condition  with  the  same 
parameters.  To  provide  a  more  realistic  model  of  an  entity,  more  physical  state 
information  is  necessary. 

Several  concepts  of  providing  entity  signature  data  within  DIS  are  being  pursued. 
The  concepts  have  included: 

•  Sending  the  signature  data  continuously  over  the  network  with  the  ESPDU. 
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•  Issuing  a  Data  Request  to  the  system  where  the  requested  entity  is  hosted,  then 
returning  the  necessary  data  to  the  requester. 

•  Inclusion  of  a  representation  index  in  the  ESPDU  to  access  a  database  of 
signature  representations  of  the  entity  of  interest. 

There  are  some  drawbacks  to  each  of  these  concepts.  The  first  and  second  concept 
involving  transmission  of  data  over  the  network  pose  significant  bandwidth  issues  since 
some  signature  representation  files  can  be  quite  large,  and  with  rapid  changes  in  the 
signature  overloading  the  network  is  probable.  Latency  issues  also  arise,  especially  in  the 
case  of  having  to  request  the  data,  and  then  waiting  to  receive  the  data  before  any 
calculation  can  be  performed.  This  is  unacceptable  for  high  performance  aircraft  or 
weapons  systems  demonstrating  search,  target  acquisition  and/or  hit  to  kill.  The 
drawback  to  the  representation  index  is  that  the  signature  data  to  be  used  during  the 
exercise  will  have  to  be  disseminated  to  participating  sites  before  the  exercise  begins. 

This  limits  the  opportunity  for  another  simulation  to  “jump”  on  the  network  to 
participate. 

It  has  been  the  goal  at  the  U.S.  Army  Test  and  Experimentation  Command 
(TEXCOM)  at  Ft.  Hunter  Liggett,  CA  to  develop  realistic  real-time  simulation  of  the 
tactical  battlefield  as  seen  through  weapons  systems  sensors.  These  efforts  are  supported 
by  the  creation  of  a  high-resolution  signature  database  and  perspective  view  generation 
research  administered  under  the  TELLUS  program  by  TRADOC  and  conducted  at  the 
Naval  Postgraduate  School  in  Monterey,  California.6  The  following  discussion  is  from 
reference  6. 

From  a  technical  point  of  view,  realistic  real-time  digital  simulation  of  the  tactical 
battlefield  presents  a  challenge.  This  is  because  the  objective  of  distributed  modeling  and 
interactive  simulation  is  to  allow  real-time  virtual  weapons  substitution  into  live  force- 
on-force  engagements  with  sufficient  sensor  fidelity  to  allow  analytical  evaluation  of 
weapon  effectiveness.  The  challenge  is  the  rapid  generation  of  metrically  accurate 
battlefield  views,  which  are  as  identical  as  possible  to  actual  sensor  displays  for  search 
and  target  acquisition  from  an  autonomous  weapon  and/or  man-in-the-loop  point  of  view. 
What’s  more,  in  order  for  systems  simulating  different  sensors  (visual  and  IR,  for 
example)  to  interoperate,  accurate  sensor  response  calculations  must  be  performed  from  a 
common  objective  representation  of  the  battlefield  and  its  environment. 

Real-time  rendering  for  simulation  of  realistic  target  signatures  in  high-resolution 
terrain  requires  the  solution  of  atmospheric  transmission  and  surface  state  calculations  in 
real  time.  This  problem  has  been  solved  through  the  use  of  rendering  look-up  tables  in 
the  TELLUS  simulator.  This  approach  divides  the  rendering  problem  into  the  off-line 
calculation  of  quantized  rendering  solutions  and  a  real  time  look-up  table  calculation  of 
sensor  spectral  response  in  a  high  speed  simulator. 

By  dividing  the  permanent  terrain  and  target  material  properties  from  the  rapidly 
varying  view  geometry  encountered  during  realistic  real-time  scene  rendering,  realistic 
rendering  speed  can  be  maintained  in  operator-controlled  joystick  vision  simulations. 

This  problem  division  allows  for  the  exchange  of  sensor  response  information  between 
distributed  simulators  using  non-real-time  data  communications  capabilities  within  the 
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High  Level  Architecture  (HLA).  To  accomplish  this  task  within  the  HLA  environment 
minimum  data  structures  must  be  defined  which  must  be  communicated  to  calculate 
realistic  real-time  sensor  signatures  of  the  same  battlefield  using  simulators  of  different 
sensors.  This  is  shown  in  Figure  6. 

Three  layers  are  shown  in  the  figure.  The  bottom  layer  is  the  Real-Time  Interface 

'  e  two  simulators.  The  middle 
layer  represents  Federation 
specific  code,  which 
automates  the  facilities, 
formats,  and  services  within  a 
Federation.  The  top  layer 
contains  the  simulators, 
which  interoperate.  The  area 
between  the  two  indicates  a 
common  data  definition 
reference  frame.  It  is  in  this 
common  data  reference  frame 
where  the  data  structures 
reside  that  perform  the 
necessary  calculations  for 

Figure  6.  HLA  Context  real-time  signature 

rendering.  This  design 

provides  the  mechanism  for  implementing  major  first  order  effects  and  represents  a  next 
step  over  other  rendering  techniques  performed  in  most  simulators. 

The  common  data  reference  frame  (see  Figure  7)  shows  five  levels  of  interrelated 
data,  which  if  exchanged,  will  allow  interacting  simulators  to  calculate  consistent  images. 
The  first  layer  is  the  terrain  or  objects  database.  This  contains  the  material  index  (M)  and 


Figure  7.  HLA  Sensor  Federation  Data  Interface  Diagram 
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incident  energy  (I)  as  a  function  of  geometric  location  X,  Y,  Z.  The  indices  M  and  I  are 
interpreted  by  the  rendering  table  (RLUT)  in  the  second  layer  to  produce  a  surface 
emitted  energy  index  (SE).  This  layer  also  requires  a  geometric  volume  grid  containing 
atmospheric  attenuation  and  radiation  index  (C)  and  a  corresponding  attenuation  table 
ATTLUT.  This  table  translates  the  emitted  surface  energy  index  into  the  sensor  input 
energy  index  SI.  The  color  lookup  table  is  required  to  translate  the  sensor  input  into  R, 

G,  B  display  intensities.  R,  G,  B  colormaps  can  easily  be  indexed  into  intensity  maps  for 
a  true  IR  sensor  display. 

This  approach  seems  to  be  headed  in  the  right  direction  for  solving  the  problems 
of  real-time  IR  scene  rendering.  Even  with  advances  in  computer  technology  and  speed, 
it’s  likely  some  level  of  off-line  calculation  of  quantized  rendering  solutions  and  look-up 
tables  will  be  needed.  This  becomes  more  apparent  as  IR  sensor  technology  matures  into 
higher  density,  higher  resolution,  and  more  sensitive  FPAs  implementing  special  signal 
processing  techniques. 

THIRD  GENERATION  IR  SENSOR  TECHNOLOGY 

As  we  move  into  the  21st  century  IR  sensor  technology  will  continue  to  grow  and 
modeling  and  simulation  of  these  new  systems  will  become  more  complex.  Efforts  are 
ongoing  to  develop  3rd  generation  IR  sensor  technology  in  order  to  satisfy  the 
requirements  of  future  search  and  target  acquisition  systems.  Basically,  3r  generation  IR 
sensor  technology  will  consist  of  large  staring  multispectral  focal  plane  arrays  and  hyper- 
spectral  sensors  with  smart  read-out  electronics  using  feedback  control  from  weapons 
system  processors  to  optimize  target  acquisition.  Justification  for  this  development  is  to. 

•  Improve  range  performance 

•  Improve  human/ ATR  interfaces 

•  Minimize  performance  degradation  from  weather  and  countermeasures 

•  Reduce  cost/weight  versus  the  current  technology 

The  U.S.  Army  NVESD  is  currently  developing  large  staring  FPA  on  the  order  of 
1280  x  960  pixels,  with  18  pm  pitch  monolithically  integrated  with  smart  silicon 
readout.7  In  comparison  to  2nd  generation  technology,  a  256  x  256  pixel  FPA  with  25 
pm  pitch  is  considered  “state-of-the-art”.  Assuming  FI  optics  and  an  aperture  diameter 
of  10  cm,  this  type  FPA  has  an  instantaneous  field  of  view  of  about  0.25  mr,  which 
results  in  approximately  0.25  meters  of  resolution  at  1  km.  Total  field  of  view  is 
approximately  3.7  degrees.  A  1280  x  960  pixel  array  with  18  pm  pitch  can  achieve  0.18 
mr,  resulting  in  0. 1 8  meters  resolution  at  1  km  range.  Total  field  of  view  is 
approximately  13  degrees  x  10  degrees.  With  respect  to  modeling  and  simulation  of 
these  3rd  generation  IR  sensor  systems,  fidelity  requirements  are  going  to  become  a 
challenge. 

In  addition  to  the  larger  higher  resolution  sensors,  these  FPAs  will  be  monolithically 
integrated  with  smart  signal  processing  capable  of  temporal,  polarization,  and 
multispectral  processing.  Temporal  processing  is  a  technique  used  to  sample  an  image  as 
a  function  of  time  and  extract  the  frequency  components  of  fluctuating  radiance.  To  do 
this  a  high  sample  frame  rate  on  the  order  of  hundreds  of  cycles  per  second  is  required. 
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This  technique  allows  for  characterization  of  objects  with  unique  cyclic  components, 
such  as  the  main  and  tail  rotor  of  a  helicopter  or  the  prop  of  an  unmanned  aerial  vehicle 
(UAV).  Another  example  of  temporal  processing  is  frame  subtraction  or  moving  target 
indication  (MTI).  This  will  allow  detection  of  objects  which  are  moving  with  respect  to  a 
stabilized  background  image. 

Polarization  signal  processing  allows  for  the  suppression  of  background  clutter.  The 
approach  is  based  on  the  difference  in  the  polarization  characteristics  in  the  thermal  IR 
from  targets  and  from  backgrounds.  The  phenomenology  is  different  but  the  signal 
processing  is  very  similar  to  the  multispectral  processing  for  two  or  three  bands.  Many 
man-made  target  surfaces  are  smooth  relative  to  5  or  10  pm,  whereas  many  natural 
materials  are  correspondingly  rough.  Most  paints  are  smooth  in  the  IR  band  and  are 
dielectric.  Metals  also  tend  to  be  smooth.  They  tend  to  be  specular  reflectors  with 
polarized  reflectances  well  described  by  the  Fresnel  equations.  Polarization  is  a  function 
of  target  surface  geometry  and  view  angle  from  the  sensor  to  the  target.  Shallow  view 
angles  to  horizontal  target  surfaces  provide  maximum  polarization,  so  polarization  is  a 
likely  candidate  processing  technique  for  clutter  suppression  in  IR  search  and  track 
systems. 

Multispectral  signal  processing  allows  for  the  collection  of  data  in  several  spectral 
bands  nearly  simultaneously  and  in  near  spatial  registration.  The  emphasis  of 
multispectral  signal  processing  is  somewhat  different  in  the  reflective  (0.4  to  2.6  pm)  and 
the  thermal  multispectral  regions  (3  to  5  and  8  to  12  pm).  In  the  reflective  region  there  is 
considerable  variability  because  of  the  variable  solar  illumination,  sun  and  view  angle 
effects,  atmospheric  scattering,  etc.  In  the  thermal  IR  it  is  temperature  variations  caused 
by  emissivity  differences  that  are  of  interest  in  discriminating  targets  from  background  or 
targets  from  targets. 

SUMMARY 

As  IR  sensor  development  has  progressed,  so  has  modeling  and  simulation  of  these 
systems.  It  has  been  argued  whether  the  detailed,  deterministic  approach  or  the  more 
general  statistical  approach  of  modeling  lends  the  most  merit  in  the  development  of 
sensor  systems.  However,  with  the  development  of  advanced  IR  systems,  it’s  not  hard  to 
see  the  complexity  and  the  challenges  we  face  with  modeling  and  simulation.  Future 
system  test  and  evaluation,  analysis,  and  training  will  depend  more  and  more  on 
modeling  and  simulation;  therefore,  it  is  imperative  we  develop  synthetic  IR  scenes  with 
as  much  fidelity  as  possible  in  order  to  accurately  model  these  advanced  3r  generation 
systems. 

As  we  move  from  DIS  and  into  the  HLA  world,  the  approaches  taken  in  generating 
synthetic  IR  imagery  must  take  into  consideration  the  anticipated  capabilities  of  3r 
generation  systems.  These  systems  will  be  able  to  perform  autonomous  target  acquisition 
and  recognition  in  addition  to,  or  assisting  the  man-in-the-loop.  It  all  boils  down  to  being 
able  to  create  the  realistic  “synthetic  battlefield”  for  input  into  the  sensor  models  in  order 
to  support  real-time  simulation  with  meaningful  results. 
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